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ABSTRACT 

A  training  system  can  only  be  effective  if  it  is  appropriately  utilized,  regardless  of  whether  the  training  system  is  a  sophisticated 
full-motion  simulator  or  steel  targets  on  a  small-arms  range.  However,  without  understanding  how  trainees  use  a  training 
system  and  without  clear  performance  feedback,  it  is  not  likely  that  desired  training  outcomes  will  be  met.  A  recent  example 
of  training-system  underutilization  impacting  training  performance  comes  from  the  U.S.  Army  Sniper  School  (USASS).  In  the 
U  SASS,  sniper  teams  spend  a  considerable  amount  of  time  at  the  beginning  of  the  course  conducting  “data  confirmation.”  Data 
confirmation  is  accomplished  by  engaging  static  targets  at  varying  distances  on  an  unknown  distance  range.  So,  in  the  case  of 
data  confirmation,  the  training  system  is  very  simple:  a  small  arms  range  with  static  targets  at  varying  distances.  It  was  observed 
over  several  iterations  of  USASS  that  shooters  rarely  engaged  targets  at  distances  beyond  600  m  during  data  confirmation  and 
that,  when  engaged,  the  hit  percentage  of  targets  over  600  m  was  very  low.  The  consequence  of  failing  to  shoot  at  far  targets 
(i.e.,  over  600  m)  during  data  confirmation  was  low  hit  percentages  on  far  targets  in  the  record  fire  event  that  was  a  graduation 
requirement.  An  intervention  was  introduced  to  increase  engagements  with  far  targets  that  required  USASS  instructors  to 
record  and  analyze  individual  shot  data.  By  requiring  instructors  to  document  data,  the  instructor  was  able  to  determine  if  the 
shooter  was  spending  too  much  time  at  closer  distances  (i.e.,  not  fully  utilizing  the  training  system)  and  to  intervene  if  necessary. 
The  result  was  increased  record-fire  performance  on  far  targets.  Even  though  the  intervention  and  results  may  seem  intuitive, 
the  need  for  such  an  intervention  highlights  the  importance  of  trainer  engagement  to  ensure  proper  training-system  utilization 
and  the  importance  of  providing  performance  feedback  during  training. 

ABOUT  THE  AUTHORS 

Martin  Bink  is  a  Senior  Research  Psychologist  and  Team  Leader  at  the  U.S.  Army  Research  Institute  for  the  Social  and  Behavioral 
Sciences  -  Fort  Benning.  Dr.  Bink’s  research  interests  are  in  human  learning,  memory,  and  cognition  especially  as  applied  to 
education  and  training,  and  his  current  research  focuses  on  digital-skills  training,  aviation-skills  training,  simulation  training  and 
adaptive  training  methods.  Dr.  Bink  holds  a  Ph.  D.  in  Cognitive  Psychology  from  The  University  of  Georgia. 

David  James  is  a  retired  Infantry  Sergeant  Major.  He  is  currently  a  Military  Trainer  with  Northrop  Grumman  Technical  Services 
supporting  the  U.S.  Army  Research  Institute  for  the  Social  and  Behavioral  Sciences  research  on  individual  and  collective  training 
and  training  effectiveness  in  both  the  live  and  virtual  environments. 

Elizabeth  Uhl  is  a  Research  Psychologist  at  the  U.S.  Army  Research  Institute  for  the  Social  and  Behavioral  Sciences  -  Fort 
Benning.  Her  current  research  interests  are  in  target  detection  and  identification,  program  evaluation,  and  the  use  of  training 
aids  and  simulators.  Dr.  Uhl  holds  a  PhD  in  General  Psychology  from  the  University  of  Texas  at  El  Paso. 


2015  Paper  No.  15209  Page  1  of  9 


Interservice/Industry  Training,  Simulation,  and  Education  Conference  (I/ITSEC)  2015 


You  Cannot  Hit  What  You  Do  Not  Shoot 


Martin  L.  Bink 
U.S.  Army  Research  Institute 
Fort  Benning,  GA 
martin.l.bink.civ@mail.mil 


David  James 

Northrop-Grumman  Corp. 
Columbus,  GA 
david.james@ngc.com 


Elizabeth  Ehl 

U.S.  Army  Research  Institute 
Fort  Benning,  GA 
elizabeth.r.uhl.civ@mail.mil 


INTRODUCTION 

A  training  system  can  only  be  effective  if  it  is  appropriately  utilized,  regardless  of  whether  the  training  system  is  a 
sophisticated  full-motion  simulator  or  steel  targets  on  a  small-arms  range.  Even  though  “utilization”  may  refer  both 
to  how  often  a  training  system  is  used  and  to  how  well  the  training  system  is  used,  utilization  is  determined  by  the 
interaction  of  the  ease  of  use  of  the  system  (e.g.,  reliability  and  familiarization)  and  the  training  value  of  the  system 
(e.g.,  skill  transfer:  Polzella,  1983).  This  paper  addresses  the  later  aspect  of  utilization,  that  is,  the  extent  to  which  the 
features  of  a  training  system  are  fully  exploited  to  achieve  training  outcomes.  A  recent  example  of  training-system 
underutilization  impacting  training  performance  comes  from  the  U.S.  Army  Sniper  School  (USASS).  In  the  USASS, 
sniper  teams  spend  a  considerable  amount  of  time  at  the  beginning  of  the  course  conducting  “data  confirmation.”  Data 
confirmation  is  accomplished  by  engaging  static  targets  at  varying  distances  on  an  unknown  distance  range.  So,  in 
the  case  of  data  confirmation,  the  training  system  is  very  simple:  a  small  arms  range  with  static  targets  at  varying 
distances.  It  was  observed  over  several  iterations  of  USASS  that  shooters  rarely  engaged  targets  at  distances  beyond 
600  m.  The  consequence  of  failing  to  shoot  at  far  targets  (i.e.,  over  600  m)  during  data  confirmation  was  low  hit 
percentages  on  far  targets  in  the  record  fire  event  that  was  a  graduation  requirement.  An  intervention  was  introduced 
to  increase  engagements  with  far  targets  and  to  provide  additional  feedback  on  all  engagements. 

Shooting  is  easy  or  so  it  seems.  The  shooter  only  needs  to  align  the  rifle  sights  with  the  target  and  squeeze  the  trigger 
without  disturbing  the  sight  alignment.  Rifle  marksmanship  research  over  the  years  has  shown  that  the  simple  task  of 
shooting  requires  complex  skill  that  involves  a  combination  of  fine  and  gross  motor  skills  coupled  with  mental 
processes  before,  during,  and  after  the  shot  (Chung,  Nagashima,  Espinosa,  Berka,  &  Baker,  2009).  Attaining  and 
refining  skills  to  consistently  and  accurately  shoot  a  rifle,  requires  hours  of  deliberate  practice.  Deliberate  practice 
requires  the  individual  to  practice  both  what  is  done  well  and  what  is  deficient  with  the  important  factor  of  focused 
feedback  (Ericsson,  Prictula,  &  Cokeley,  2007). 

In  general,  marksmanship  training  systems  consist  of  a  range  (live  or  virtual)  that  allows  shooters  to  engage  targets  at 
a  variety  of  distances.  Additional  complexity  can  be  added  by  the  capability  to  shoot  from  alternate  positions  (e.g., 
loop  hole  or  elevated  position),  to  engage  moving  targets,  to  engage  partially  obscured  targets  or  targets  of  varying 
size,  and  to  provide  combat  situations  (e.g.,  time  pressure  or  distracting  sounds).  In  a  sense,  deliberate  practice 
requires  individuals  to  reinforce  existing  skill  and  move  beyond  the  comfort  zone  in  order  to  expand  skill  application. 
Accordingly,  a  marksmanship  training  range  must  facilitate  practice  on  both  easier  engagements  (e.g.,  close  targets) 
and  more  difficult  engagements  (e.g.,  far  targets).  The  simplicity  of  a  marksmanship  range  (i.e.,  an  open  range  with 
targets  at  varying  distance)  does  not  inherently  provide  effective  training.  Rather,  the  value  of  the  marksmanship 
range  is  in  the  deliberate -practice  approach  used  on  the  range. 

Initial  observations  of  USASS  students  conducting  data  confirmation  identified  the  application  of  unguided  practice 
with  limited  feedback  during  training  rather  than  deliberate  practice.  The  primary  purpose  of  data  confirmation  is  to 
confirm  or  adjust  the  ballistic  calculations  for  the  weapon  system  at  different  distances,  but  data  confirmation  also 
provides  an  opportunity  for  shooters  to  practice  static  target  engagements  and  to  practice  shooter-spotter  teamwork. 
Shooters  engage  a  target  at  a  given  distance  until  the  target  is  consistently  hit  and  then  repeat  the  process  with  a  target 
at  a  different  distance.  The  typical  pattern  of  target  engagements  observed  during  data  confirmation  was  to  start  with 
close  targets  and  frequent  hits,  move  to  far  targets  with  infrequent  hits,  and  then  move  back  to  close  targets  and  shoot 
a  lot  more.  This  pattern  resulted  in  spending  too  much  time  at  closer  distances  without  the  ability  to  diagnose  and 
remediate  problems  with  shooting  performance.  In  concert  with  USASS  instructors,  an  intervention  was  developed 
that  focused  on  skills  required  to  engage  long-range  targets  while  at  the  same  time  providing  a  forcing  function  to 
promote  interaction  between  the  student  sniper  team  (i.e.,  shooter  and  spotter)  and  the  sniper  instructor. 
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United  States  Army  Sniper  Course  Data  Confirmation 

The  USASS  trains  Soldiers  to  engage  single  stationary  and  moving  targets  from  300  to  1200  meters  using  both 
unknown  and  known  distance  ranges.  Sniper  training  is  team  training.  That  is,  students  train  as  two-person  teams  and 
are  trained  in  both  shooter  and  spotter  roles.  Student  sniper  teams  train  on  target  acquisition,  range  determination, 
and  engagements  technique  skills  with  an  explanation  of  how  to  divide  these  responsibilities  between  team  members 
while  engaging  a  target.  Both  team  members  must  be  proficient  on  all  tasks  as  they  will  alternate  roles  throughout  the 
different  training  tasks.  Teamwork  skills  are  trained  as  a  precursor  to  the  live-fire  training  and  practiced  during  data 
confirmation. 

Data  confirmation  in  the  USASS  is  the  process  of  engaging  targets  at  unknown  distances  to  verify  firing  data  through 
proper  utilization  of  the  advanced  ballistic  calculator  for  future  engagements.  The  data-confirmation  process  involves: 

•  Accurately  recording  cold  barrel  and  confirmation  shot  placement,  elevation  and  windage  adjustments  and 
holds. 

•  Properly  identifying  targets. 

•  Accurately  determining  range  to  targets. 

•  Accurately  utilizing  the  advanced  ballistic  calculator. 

•  Accurately  engaging  targets  with  precision  fire. 

Data  confirmation  is  conducted  daily  during  the  live-fire  portion  of  the  course  and  culminates  with  a  record  fire  event 
that  the  Soldiers  must  pass  to  graduate  USASS.  Data  confirmation  is  conducted  on  an  unknown  distance  range  with 
steel  targets  randomly  placed  between  300  and  1000  meters.  Numbers  and  letters  are  painted  on  the  targets  to  enable 
identification  and  are  moved  to  a  different  location  on  the  range  prior  to  each  training  day.  The  sniper  teams  are  taught 
a  “shot  process”  to  engage  targets  during  data  confirmation.  Figure  1  depicts  the  shot  process  the  team  uses  for  each 
round  fired.  During  the  unguided  practice,  each  sniper  team  picked  the  first  and  subsequent  targets  to  engage. 


The  Shooter 
measures  the  target 
with  the  rifle  scope 
reticle  and  relays 
the  results  to  the 
Spotter 


The  Shooter  dials 
the  MOA  on  the 
rifle  scope 


The  Spotter  inputs 
the  measurements 
into  the  advanced 
ballistic  calculator 
(ABC) 


The  Spotter 
determines  wind 
direction  and  speed 
through  his  spotting 
scope 


The  ABC 
approximates  the 
range  to  the  target 


The  Spotter  makes 
a  "wind  call"  by 
announcing  how  far 
right  or  left  of 
target  the  point  of 
aim  should  be 


The  Spotter  relays 
the  distance  and 
minutes  of  angle 
(MOA)  back  to  the 
Shooter 

I 


The  Shooter  aims 
where  the  Spotter 
tells  him  and  fires 


The  Spotter 
watches  the  flight 
and  impact  of  the 
round 


If  the  target  is  hit, 
the  team  moves  to 
the  next  target 


If  the  target  is 
missed,  the  team 
makes  corrections 
and  re-engages 


Figure  1.  The  shot  process  used  by  the  sniper  teams  to  engage  each  target  during  unguided  practice. 
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THE  UTILIZATION  INTERVENTION 

Again,  the  initial  data-confirmation  observations  identified  that,  for  the  most  part,  the  teams  were  conducting  unguided 
practice  with  limited  feedback  from  the  USASS  instructors.  It  was  also  observed  that  errors  made  in  the  range 
determination  and  engagement  technique  process  were  not  mitigated  between  shots  and  that  when  teams  missed  at 
longer  ranges,  they  tended  to  fall  back  into  their  comfort  zone  and  only  engage  shorter  range  targets  that  would  give 
them  positive  feedback  (i.e.,  the  “ding”  of  the  round  hitting  steel).  The  utilization  intervention  required  instructors  to 
document  students’  shot  data.  By  doing  so,  the  instructor  was  able  to  determine  if  the  shooter  was  spending  too  much 
time  at  closer  distances  (i.e.,  not  filly  utilizing  the  training  system)  and  to  intervene  if  necessary. 

The  utilization  intervention  was  designed  as  an  exercise  in  deliberate  practice  that  focused  on  the  skills  required  to 
engage  long  range  targets.  Implementing  the  intervention  capitalized  on  a  tool  that  was  familiar  to  the  USASS 
instructors  and  already  available  to  USASS  students.  Shot  data  books  are  an  analog  tool  used  to  record  environmental 
conditions  (e.g.,  temperature,  humidity,  etc.),  range  to  targets,  engagement  techniques  (i.e.,  hold-overs,  dial,  or  rapid 
target  engagement),  wind  direction  and  speed,  wind  hold,  and  location  of  hits  and  misses  for  each  round  fired  during 
training.  The  purpose  of  the  shot  data  book  is  to  capture  relevant  historical  shooting  information  that  can  be  referred 
to  for  data  on  previous  engagements  and  past  performance  gaps.  All  USASS  students  are  provided  hard  copies  of  a 
shot  data  book  as  part  of  the  student  handouts  at  the  beginning  of  the  course.  However,  little  emphasis  was  placed  on 
using  data  books  during  data  confirmation  as  more  emphasis  was  placed  on  using  the  digital  advanced  ballistic 
calculator. 

For  the  utilization  intervention,  a  simplified  shot  data  book  was  re-introduced.  These  modified  shot  data  books 
included  only  range  to  target,  engagement  technique,  and  hit  or  miss.  The  purpose  of  the  simplified  data  book  was  to 
enable  both  the  student  sniper  team  and  the  instructor  to  visualize  shooting  performance  during  and  af er  the  day’s 
training.  The  data  book  was  coupled  with  recommendations  on  how  the  sniper  instructor  should  use  the  book  while 
providing  focused  feedback  during  the  shot  process.  As  opposed  to  the  unguided  practice  identified  for  typical  data 
confirmation,  the  utilization  intervention  required  the  sniper  instructor  to  facilitate  deliberate  practice  by  choosing  all 
targets  as  well  as  providing  feedback  at  critical  points  in  the  shot  process  (see  Figure  2).  After  the  completion  of  each 
successful  engagement,  the  sniper  instructor  would  select  the  next  target  for  the  student  sniper  team  to  insure  that 
targets  at  all  range  bands  -  near  and  far  -  were  engaged.  The  instructor  also  documented  each  shot  in  the  team’s  data 
book  in  order  to  review  performance  and  determine  strengths  and  weaknesses  to  tailor  the  next  day’s  training. 


METHOD 

Participants 

Data  were  collected  from  six  (6)  USASS  classes  with  a  total  of  224  Soldiers.  The  USASS  students  ranged  in  rank 
from  Private  First  Class  to  Sergeant  First  Class.  The  classes  were  conducted  from  March  through  August  during  a 
recent  training  year.  Two  classes  were  simultaneously  conducted  in  each  of  three  cycles  over  this  time  period.  At 
the  outset  of  each  class,  USASS  students  were  placed  into  two-man  sniper  teams.  Existing  sniper  teams  were  kept 
intact,  and  Soldiers  who  arrived  individually  were  partnered  with  another  student.  In  the  case  of  an  uneven  number  of 
Soldiers  in  a  class,  three-man  sniper  teams  were  formed.  USASS  instructors  were  assigned  mentor  groups  in  each 
class  with  the  goal  of  an  instructor-student  ratio  of  1 :4  (i.e.,  one  instructor  per  two  sniper  teams). 

Procedure 

Each  course  was  conducted  in  accordance  with  the  training  program  of  instruction  with  the  exception  of  the  utilization 
intervention  during  the  second  cycle  of  two  classes.  The  utilization  intervention  previously  described  was  only  used 
during  the  firing  events  that  led  to  the  first  record  fire.  After  that,  no  modification  of  the  course  occurred.  In  order  to 
provide  a  firm  baseline  and  to  further  isolate  the  effect  of  the  utilization  intervention,  the  application  of  the  intervention 
followed  an  ABA  design.  Data  was  collected  from  three  consecutive  cycles  of  two  classes.  The  first  cycle  of  two 
classes  (i.e.,  Class  5  &  6)  were  not  given  the  utilization  intervention  and  merely  allowed  students  to  conduct  data 
confirmation  without  instructor  input  (i.e.,  initial  baseline).  During  the  second  cycle  of  two  classes  (i.e.,  Class  7  &  8), 
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instructors  applied  the  utilization  intervention  for  all  students.  The  final  cycle  of  two  classes  (i.e.,  Class  9  &  10) 
received  no  intervention  just  as  the  first  cycle  (i.e.,  return  to  baseline). 

USASS  instructors  were  provided  an  introduction  to  the  utilization  intervention  prior  to  Class  7  &  8.  The  modified 
shot  data  books  were  distributed  to  each  instructor.  A  member  of  the  research  team  was  present  during  the  first  two 
days  of  data  confirmation  to  ensure  the  intervention  was  properly  applied  and  to  answer  questions  from  the  instructors. 
Thereafter,  the  research  team  “spot  checked”  the  intervention  and  retrieved  data  books  every  other  day  during  data 
confirmation.  The  procedure  for  the  application  of  the  utilization  intervention  is  given  in  Figure  2. 

The  static-target  record  fire  event  for  all  classes  was  conducted  according  to  the  course  program  of  instruction,  and 
data  (i.e.,  record  fire  scores)  were  provided  to  the  research  team  from  USASS  instructors.  Record  fire  is  a  graded 
event.  Each  member  of  the  student  sniper  teams  engages  five  targets  that  vary  in  range  from  350  m  to  800  m.  The 
members  of  the  team  alternate  shooter  and  spotter  positions  during  the  event.  A  first-round  hit  on  a  target  is  scored 
as  10  points,  a  second -round  hit  on  a  target  is  scored  as  5  points,  and  0  points  are  scored  for  a  second-round  miss. 
Thus,  each  shooter  can  obtain  a  maximum  score  of  50  points  on  the  record  fire  event. 


Instructor  announces 
the  target  to  engage 


Instructor  confirms 
correct  range  to  target. 
If  in  error,  announces ™ 
“Re-measure”  and  talks 
about  range  estimation 
process. 

Instructor  confirms 
the  wind  call  If  in 
error,  talks  to  spotter 
about  reading  wind. 


Instructor  provides 
feedback  on  the 
shooters 
marksmanship 
fundamentals 


The  Shooter 
measures  the  target 
with  the  rifle  scope 
reticle  and  relays 
the  results  to  the 
Spotter 


-> 


The  Spotter  inputs 
the  measurements 
into  the  advanced 
ballistic  calculator 
(ABC) 


The  ABC 
approximates  the 
ranae  to  the  taraet 


1  The  Spotter  relays 

The  Spotter 

L.  the  distance  and 

The  Shooter  dials 

determines  wind 

minutes  of  angle 

—> 

the  MOA  on  the  — > 

direction  and  speed 

(MOA)  back  to  the 

rifle  scope 

throuah  his  spottina 

Shooter 

scope 

\T 


The  Spotter  makes 
a  "wind  call"  by 
►announcing  how  far 
right  or  left  of 
target  the  point  of 
aim  should] 


> 


The  Shooter  aims 
where  the  Spotter 
tells  him  and  fires 


The  Spotter 
watches  the  flight 
and  impact  of  the 
round 


If  the  target  is  hit, 
the  team  moves  to 
the  next  taraet 


If  the  target  is 
missed,  the  team 
makes  corrections 
and  re-enaaaes 


Instructor  stops 
the  exercise  and 
discusses  what 
happened 


Figure  2.  The  shot  process  used  by  the  sniper  teams  to  engage  each  target  during  the  utilization  intervention. 
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RESULTS 

The  primary  data  consisted  of  proportion  of  shots  and  hits  for  near  targets  (i.e.,  less  than  500  m)  and  far  targets  (i.e., 
greater  than  600  m)  during  data  confirmation  and  record  fire.  The  data  come  from  all  rounds  fired  on  all  events  from 
all  students.  Because  data  confirmation  target  engagements  are  self-determined,  each  student  fired  different  number 
of  rounds  at  different  ranges  at  different  times  and  in  different  orders.  As  such,  there  was  no  way  of  constructing 
viable  statistical  analyses  for  the  shot  data.  Instead,  graphical  analyses  were  used  to  determine  the  pattern  of  results. 
If  the  utilization  intervention  produced  the  desired  effect  and  if  the  effect  was  isolated  to  the  intervention,  then  data 
values  for  Class  7  &  8  should  be  greater  than  both  Class  5  &  6  and  Class  9  &  10.  The  graphical  analyses  were  used 
to  determine  if  the  utilization  intervention  resulted  in  (a)  more  engagements  at  far  targets  and  (b)  more  accuracy  (i.e., 
hits)  on  far  targets  than  the  self-guided  training.  In  addition,  scores  from  record  fire  were  statistically  analyzed  to 
determine  the  impact  of  the  utilization  intervention  on  course  performance.  Finally,  the  proportion  of  shots  and  hits 
for  targets  greater  than  700  m  were  graphically  analyzed  to  determine  if  the  training  utilization  intervention  had  an 
impact  on  the  most  difficult  skill  application. 

The  proportion  of  shots  for  near  targets  and  far  targets  during  data  confirmation  were  combined  into  a  ratio  and  plotted 
to  determine  if  the  intervention  produced  broader  utilization  of  the  available  targets.  The  proportion  of  far-target  shots 
was  calculated  as  a  function  of  the  proportion  of  near-target  shots.  This  ratio  expresses  the  distribution  of  shots  at  the 
two  range  bands.  Ratios  closer  to  1.00  indicate  an  even  distribution  of  shots  across  the  two  range  bands,  ratios  under 
1.00  indicate  that  more  shots  were  fired  at  near  targets  then  at  far  targets,  and  ratios  greater  than  1.00  indicate  that 
more  shots  were  fired  at  far  targets  than  at  near  targets.  An  inspection  of  the  blue  bars  in  Figure  3  clearly  indicates 
that  only  students  in  Class  7  &  8  (.i.e.,  the  students  with  the  utilization  intervention)  had  an  even  distribution  of  shots 
across  far  targets  and  near  targets  (i.e.,  ratio  approximately  equal  to  1.00).  Class  5  &  6  and  Class  9  &  10  shot  more 
rounds  at  near  targets  than  at  far  targets  (i.e.,  ratio  substantially  less  than  1.00). 


Figure  3.  Ratio  of  far-target-to-near-target  proportions  for  shots  during 
data  confirmation  and  for  record  fire  hits. 

The  set  of  gray  bars  in  Figure  3  shows  the  far-target-to-near-target  ratios  for  the  proportion  of  hits  on  record  fire.  As 
can  be  seen,  all  classes  hit  proportionately  fewer  far  targets  than  near  targets.  Flowever,  Class  7  &  8  had  a  greater  far- 
target-to-near-target  ratio  than  either  Class  5  &  6  or  Class  9  &  10. 

The  mean  individual  record  fire  scores  for  all  three  USASS  class  groups  were  compared  in  a  one-way  analysis  of 
variance.  Student-Newman-Kuels  post-hoc  comparisons  were  used  to  determine  the  pattern  of  effects.  The  analysis 
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was  conducted  at  the  5-percent  alpha-error  rate.  The  mean  record-fire  scores  for  Class  7  &  8  (mean  =  36.36,  SE,„  = 
0.93)  was  statistically  higher  than  the  mean  scores  for  Class  5  &  6  (mean  =  29.76,  SEm  =  1.01)  and  mean  scores  for 
Class  9  &  10  (mean  =  27.15,  SEm  =  1.14),  F  (2,  221)  =  20.18,  MSe  =  79.33,  p  <  .01.  This  result  indicated  that  the 
utilization  intervention  increased  overall  static -target  performance. 

The  final  set  of  analyses  focused  on  performance  for  the  most  distant  target  range,  i.e.,  greater  than  700  m.  Because 
hitting  targets  at  this  range  is  more  difficult,  this  analysis  provided  a  stricter  test  of  the  impact  of  the  utilization 
intervention.  This  set  of  analyses  again  examined  the  proportion  of  shots  during  data  confirmation  and  hits  on  record 
fire.  In  addition,  the  proportion  of  hits  during  data  confirmation  was  added  to  determine  if  the  intervention  also 
impacted  training  performance.  Because  of  the  nature  of  the  data,  graphical  analysis  again  was  used  to  determine  the 
pattern  of  effects.  The  left-hand  set  of  bars  in  Figure  4  show  the  proportions  of  shots  during  data  confirmation  for 
each  USASS  class  group.  While  all  three  bars  of  this  set  are  nearly  equivalent,  the  orange  bar  indicates  that  Class  7 
&  8  (i.e.,  the  utilization  intervention)  shot  a  higher  proportion  of  data  confirmation  rounds  at  targets  farther  than  700 
m  than  did  Class  5  &  6  or  Class  9  &  10.  The  middle  set  of  bars  in  Figure  4  show  that  Class  7  &  8  also  had  a  higher 
proportion  of  hits  on  targets  farther  than  700  m  than  did  Class  5  &  6  or  Class  9  &  10.  Taken  together,  these  two  results 
indicated  that  the  students  who  received  the  utilization  intervention  were  more  efficient  at  engaging  targets  farther 
than  700  m  than  students  who  were  self-guided  in  these  target  engagements.  The  right-hand  set  of  bars  in  Figure  4 
show  that  the  intervention  resulted  in  a  higher  proportion  of  hits  on  targets  farther  than  700  m  during  record  fire  than 
did  the  other  groups  of  students.  This  result  again  shows  that  the  utilization  intervention  during  training  translated  to 
higher  performance  on  the  graded  event. 


Figure  4.  Proportions  for  shots  and  hits  on  targets  farther  than  700  m  during 
data  confirmation  and  record  fire. 


DISCUSSION 

The  purpose  of  the  research  was  to  increase  the  utilization,  and  thereby  the  effectiveness,  of  a  very  simple  training 
system.  The  results  converged  on  the  conclusion  that  the  intervention  produced  more  efficient  utilization  of  the 
training.  USASS  students  who  received  the  intervention  (a)  shot  at  more  far  targets  during  data  confirmation,  (b)  had 
greater  accuracy  on  far  targets  during  data  confirmation,  and  (c)  obtained  higher  record  fire  scores  than  students  who 
were  not  exposed  to  the  intervention.  The  intervention  focused  on  improving  both  how  much  of  the  training  system 
was  utilized  (i.e.,  instructors  announced  the  targets  to  engage)  and  how  well  the  training  system  was  utilized  (i.e., 
instructors  provided  feedback  and  discussion  on  target  engagements).  Figure  4  provides  a  good  illustration  of  how 
the  intervention  had  its  desired  effect.  Students  who  received  the  intervention  proportionally  shot  a  few  more  rounds 
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at  targets  greater  than  700  m.  That  small  increase  in  number  of  rounds  (i.e.,  “how  much”)  was  coupled  with  a  slightly 
larger  increase  in  accuracy  (i.e.,  “how  well”),  which  translated  to  about  a  10%  increase  in  record-fire  hits  (i.e.,  training 
effectiveness)  at  targets  greater  than  700  m.  In  other  words,  if  USASS  students  did  not  effectively  engage  far  targets 
in  training,  then  students  did  not  effectively  hit  far  targets  on  the  test  (i.e.,  record  fire).  Even  though  the  intervention 
and  results  may  seem  intuitive,  the  need  for  such  an  intervention  highlights  the  importance  of  trainer  engagement  to 
ensure  proper  training-system  utilization  and  the  importance  of  providing  performance  feedback  during  training 
(Blaiwes,  Puig,  &  Regan,  1973). 

The  utilization  intervention  was  based  on  deliberate  practice  techniques.  Deliberate  practice  requires  effortful 
repetitions  with  formative  feedback,  is  engaging,  and  leverages  pre-existing  knowledge  to  improve  skill  efficiency 
(i.e.,  speed  and  accuracy)  and  retention  (Ericsson,  Krampe,  &  Tesch-Romer,  1993).  Implicit  in  this  definition  of 
deliberate  practice  is  the  notion  that  the  role  of  practice  is  to  continually  progress  skill  performance.  As  such,  practice 
must  both  exercise  skills  at  the  “edge  of  failure”  as  well  as  reinforce  existing  skills.  Practicing  skills  at  the  edge  of 
failure  requires  refinement  and  exploration  of  skill  application,  allows  opportunities  for  formative  feedback,  and 
provides  an  engaging  context  without  overwhelming  the  cognitive  resources  of  the  individual  (Ali,  Guckenberger, 
Rossi,  &  Williams,  2000;  Hancock,  2009).  Maintaining  the  edge  of  failure  is  achieved  through  the  controlled 
progression  of  difficulty  with  performance  support  as  difficulty  increases  (Hogan  &  Pressley,  1997).  Of  course, 
feedback  is  used  to  calibrate  the  difficulty  of  the  learning  task  and  to  adjust  the  level  of  support  (i.e.,  fading).  In  the 
case  of  the  utilization  intervention  reported  here,  information  from  the  shot  data  books  and  the  expertise  of  the  U  SASS 
instructors  provided  the  basis  for  feedback  and  controlled  progression. 

Interestingly,  the  ABA  research  design  not  only  provided  additional  experimental  control  for  the  data,  but  also  allowed 
inferences  about  utilization  challenges.  The  difference  in  far-target  engagements  and  record-fire  scores  between 
baseline  (i.e.,  Class  5  &  6)  and  intervention  (i.e.,  Class  7  &  8)  represented  a  training  challenge.  That  is,  there  was  a 
performance  gap  that  was  addressed  with  a  training  solution.  In  this  case,  an  instructional  intervention.  The  difference 
in  far-target  engagements  and  record-fire  scores  between  intervention  (i.e.,  Class  7  &  8)  and  return  to  baseline  (i.e., 
Class  9  &  10)  represented  a  Leadership  challenge.  According  to  the  research  design,  instructors  were  not  required  to 
use  the  modified  shot  data  books  or  to  engage  students  during  data  confirmation.  However,  there  were  no  controls  to 
prevent  instructors  from  using  techniques  or  procedures  during  Class  9  &  10  that  they  previously  learned  from  the 
utilization  intervention.  That  is,  it  was  expected  that  there  would  be  some  carryover  effects  of  the  intervention  in  the 
return  to  baseline  phase,  but  the  data  clearly  demonstrated  that  was  not  the  case.  The  lack  of  carryover  highlights  the 
importance  of  Leaders,  course  mangers,  and  training-system  operators  emphasizing  effective  utilization  techniques 
and  preparing  instructors  and  trainers  to  fully  utilize  the  training  systems. 

Even  though  the  current  research  addressed  a  simple  training  system,  the  findings  may  generalize  to  more  complex 
training  systems.  There  is  a  tendency  to  believe  that  training  effectiveness  improves  as  the  training  systems  become 
more  technologically  sophisticated.  There  is  also  the  belief  that  simply  using  a  training  system  provides  effective 
training.  However,  sophistication  does  not  always  yield  effectiveness  and  training  systems  do  not  always  replace 
training  (e.g.,  Salas,  Bowers,  &  Rhodenizer,  1998;  Stewart,  Johnson,  and  Howse,  2008).  Instead,  it  is  important  to 
keep  in  mind  that  training  systems  are  merely  tools  to  enable  effective  and  efficient  training  and  that  skill  development 
is  the  result  of  the  process  of  optimally  applying  training  tools  and  training  techniques. 
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